Sensorimotor impairment and haptic support in microgravity

Future space missions envisage human operators teleoperating robotic systems from orbital spacecraft. A potential risk for such missions is the observation that sensorimotor performance deteriorates during spaceflight. This article describes an experiment on sensorimotor performance in two-dimensional manual tracking during different stages of a space mission. We investigated whether there are optimal haptic settings of the human-machine interface for microgravity conditions. Two empirical studies using the same task paradigm with a force feedback joystick with different haptic settings (no haptics, four spring stiffnesses, two motion dampings, three masses) are presented in this paper. (1) A terrestrial control study (N = 20 subjects) with five experimental sessions to explore potential learning effects and interactions with haptic settings. (2) A space experiment (N = 3 cosmonauts) with a pre-mission, three mission sessions on board the ISS (2, 4, and 6 weeks in space), and a post-mission session. Results provide evidence that distorted proprioception significantly affects motion smoothness in the early phase of adaptation to microgravity, while the magnitude of this effect was moderated by cosmonauts' sensorimotor capabilities. Moreover, this sensorimotor impairment can be compensated by providing subtle haptic cues. Specifically, low damping improved tracking smoothness for both motion directions (sagittal and transverse motion plane) and low stiffness improved performance in the transverse motion plane

KONTUR-2: Force-feedback Teleoperation from the International Space Station

This paper presents a new robot controller for space telerobotics missions specially designed to meet the requirements of KONTUR-2, a German & Russian telerobotics mission that addressed scientific and technological questions for future planetary explorations. In KONTUR-2, Earth and ISS have been used as a test-bed to evaluate and demonstrate a new technology for real-time telemanipulation from space. During the August 2015' experiments campaign, a cosmonaut teleoperated a robot manipulator located in Germany, using a force-feedback joystick from the Russian segment of the International Space Station (ISS). The focus of the paper is on the design and performance of the bilateral controller between ISS joystick and Earth robot. The controller is based on a 4-Channels architecture in which stability is guaranteed through passivity and the Time Delay Power Network (TDPN) concept. We show how the proposed approach successfully fulfills mission requirements, specially those related to system operation through space links and internet channels, involving time delays and data losses of different nature

Model-Augmented Haptic Telemanipulation: Concept, Retrospective Overview, and Current Use Cases

Certain telerobotic applications, including telerobotics in space, pose particularly demanding challenges to both technology and humans. Traditional bilateral telemanipulation approaches often cannot be used in such applications due to technical and physical limitations such as long and varying delays, packet loss, and limited bandwidth, as well as high reliability, precision, and task duration requirements. In order to close this gap, we research model-augmented haptic telemanipulation (MATM) that uses two kinds of models: a remote model that enables shared autonomous functionality of the teleoperated robot, and a local model that aims to generate assistive augmented haptic feedback for the human operator. Several technological methods that form the backbone of the MATM approach have already been successfully demonstrated in accomplished telerobotic space missions. On this basis, we have applied our approach in more recent research to applications in the fields of orbital robotics, telesurgery, caregiving, and telenavigation. In the course of this work, we have advanced specific aspects of the approach that were of particular importance for each respective application, especially shared autonomy, and haptic augmentation. This overview paper discusses the MATM approach in detail, presents the latest research results of the various technologies encompassed within this approach, provides a retrospective of DLR's telerobotic space missions, demonstrates the broad application potential of MATM based on the aforementioned use cases, and outlines lessons learned and open challenges

Comparing the effects of space flight and water immersion on sensorimotor performance

Several studies documented the detrimental effects of microgravity during spaceflight on human motor control e.g. during aiming tasks. In addition to parabolic flight, water immersion has been used for simulating microgravity effects on earth. Until now, however, the validity of partial or full water immersion setups as test environments to explore effects on sensorimotor performance has not been tested. In the present paper, the results of three empirical studies were compared using the identical aiming task paradigm during forearm water immersion (N = 19), full body water immersion (N = 22), and during spaceflight (N = 3 astronauts). In line with prior research, slower aiming motion profiles were found during spaceflight (2 weeks in space) compared to the terrestrial experiments. Astronauts required substantially more time to approach target areas for the desired level of target matching precision in space. Average motion speed and speed variance decreased significantly. Intriguingly, the same overall effect pattern was evident in both partial and full water immersion, although the effect sizes tended to be smaller. Altogether, results indicate that water immersion is a valid form of weightlessness simulation. However, effects solely present during spaceflight (such as vestibular dysfunction) additionally contribute to performance losses

Disentangling the Enigmatic Slowing Effect of Microgravity on Sensorimotor Performance

The success of many space missions depends on astronauts’ performance. Yet, prior research documented that sensorimotor performance is impaired in microgravity, e.g. aimed arm movements are slowed down and are less accurate. Several explanatory approaches for this phenomenon have been discussed, such as distorted proprioception or stress-related attentional deficits. In the current work, sensorimotor performance was investigated during aimed joystick-controlled motions in a simulation. The task included rapid as well as fine matching motions. Results of two different studies were compared: 1) a study utilising a dual-task paradigm to investigate the impact of attentional distraction (N = 19) and 2) a study investigating the impact of microgravity during spaceflight (N = 3). In both studies, an overall slowing effect was found. However, results diverged when comparing feedforward vs. feedback controlled parts of aiming. Reduced attentional resources mainly affected feedforward control, which was reflected in significantly longer response times and longer rapid motion times. Microgravity, however, did not affect response times at all, but rapid aiming times as well as fine matching times substantially increased. These findings provide evidence that impaired attention is not the main trigger behind the slowing effect, but rather it is distorted proprioception which impairs feedback controlled motions

Teleoperating Robots from the International Space Station: Microgravity Effects on Performance with Force Feedback

Sending humans to Mars’ surface to build habitats is, for now, prohibitively dangerous and costly. An alternative is to have humans in orbiters, teleoperating robots on Mars to construct habitats, deferring human arrival until these habitats are finished. This paper describes the Kontur-2 experiments, in which the feasibility of this scenario was tested with the International Space Station as an orbiter, a cosmonaut operating a force-feedback joystick as an input device for teleoperation, and Earth as the planet where the teleoperated robot is located. In particular, we focus on human teleoperation performance, which is known to deteriorate under conditions of spaceflight. We investigate whether the provision of force feedback at the joystick is as beneficial as under terrestrial conditions. Our results show that, to support humans operating in weightlessness, haptic assistance needs to be adjusted to the altered environmental condition

Haptic intention augmentation for cooperative teleoperation

Multiple robotic agents, autonomous or teleoperated, can be employed to synergise and cooperate to achieve a common objective more effectively. Tasks using robotic manipulators can be eased and improved in terms of reliability, adaptability and ergonomics via robot cooperation. In spite of visual and haptic feedback, cooperative telemanipulation of multiple robots by distant operators can still be challenging due to practical limitations in synchronisation and supervision. This paper presents a new control approach for haptic intention augmentation between two human operators handling objects via teleoperation in a cooperative manner. The force feedback to each operator is enhanced by information on the motion intention of the other operator observed by a force sensor at the input devices. Besides on-ground experiments, an experiment is presented that involves the cooperative teleoperation of an on-ground robot by a cosmonaut on the International Space Station and another distant operator on ground

Kontur-3: Human Machine Interfaces for Telenavigation and Manipulation of Robots from ISS

Space agencies are planning crewed lunar and Mars exploration missions to be realized within the next decade [1]. Sending humans directly to the surface of the celestial bodies is, however, extremely dangerous and costly. Therefore, humans will teleoperate robots from an orbital spacecraft to explore the surface, acquire samples and to construct habitats. In the German-Russian Kontur-2 space project (2012-2017), the basic telerobotic scenario was realized, i.e. cosmonauts located in an orbiter (ISS) controlled robotic systems on a planet (Earth) with a force feedback joystick. This paper presents the planned experiments and Human Machine Interface (HMI) devices for the Kontur-3 space project, which is a follow-up project of Kontur-2. The paper reports the main technical achievements and human factors results of the Kontur-2 project and, the need for more effective HMI devices to be developed in Kontur-3. The preparations for the ISS experiments with the newly developed devices would bring new insights to the research community and can also become a major contribution to the planning, development and implementation of future space missions

Force-feedback teleoperation of on- ground robots from the international space station in the frame of the KONTUR-2 experiment

The issues on creation and using of the haptic interface for remote control of on-ground robots from the Russian Segment of the International Space Station (ISS RS) in the frame of “KONTUR-2” space experiment are presented. Force-feedback as key technology of this system ensures elements of telepresence of operator in the environment where robot operates using visual and tactile feedback in a closed control loop. Results of space sessions on control of on-ground robots from the ISS RS are presented